Segmentation, Index and Summarization of Digital Video Content

Segmentation, Index and Summarization of Digital Video Content Di Zhong In this thesis, we propose and develop unique frameworks and methods for temporal and spatial video segmentation as well as object based video representation, indexing and retrieval at both the syntactic and the semantic level. First we demonstrate a robust and real-time temporal scene cut detection system that combines color, edge and motion features in both compressed and uncompressed domains. Contrary to existing work, we considered comprehensive issues in practical situations, such as gradual transition, lighting change and motion. The algorithms perform very well for variant kinds of videos, including sports, sitcom, news, cartoon, movie and home videos. Then we present an automatic region segmentation system for content-based video search. The system segments and tracks consistent regions through each video shot, and then computes visual features of extracted regions to build visual libraries that support region level search. A web-based video query system that has more than 3,000 video shots has been built. The query system allows users to do spatial-temporal search of video shots by drawing regions and specifying features. It is the first video search engine that supports automatic extraction and object-level motion-based search. Semantic object segmentation and tracking is then studied to produce high-level object